Obstacle Detection on Railway Track by Fusing Radar and Image Sensor

Novel technology to recognize the situation in distant place is necessary to develop a railway safety monitoring system by which human being having fallen onto the tracks from a platform and obstacles in the level crossing can be detected. In this research, we propose a method for detecting a stationary or moving obstacle by the technology which employs the super resolution radar techniques, the image recognition techniques, and the technology to fuse these techniques. Our method is designed for detecting obstacles such as cars, bicycles, and human on the track in the range up to hundreds of meters ahead by using sensors mounted on a train. In super resolution radar techniques, novel stepped multiple frequency radar is confirmed to provide the expected high resolution performance by experimental study using software defined radar. We designed the parameters of the stepped multiple frequency radar. The signal processing processor which employs A/Ds, D/As of high-sampling rate and the highest performance FPGAs has been designed and developed. In image recognition techniques, the main algorithm which detects a pair of rails is based on information about rail edges and intensity gradients. As the obstacle exists in the vicinity of rails, all the detection processes can be limited to the region in the vicinity of rails. As an example, we detect an obstacle by estimating image boundaries which enclose a group of feature points exactly on the obstacle. In order to determine the group of feature points, feature points of the whole image are detected at each frames and tracked over image sequence. For robustness estimation of boundary, we use radar so as to detect the obstacle’s rough position in an image region where an obstacle would exist; the motion segmentation technique is applied to the tracks that is located in the region. Note that an obstacle's position detected by radar is remarkably rough because radar has low transverse resolution. Nevertheless, the position detected by radar contributes to rapid and robust estimation of satisfactory image boundaries. To allow for monitoring in large distances, a prototype for an onboard pan/tilt camera platform with directional control was designed and manufactured as required in a railway situation. To detect obstacles in real time, we have built a high-performance test machine with a GPU and an image processing environment.